Deep neural networks (DNNs) achieve remarkable performance on a wide range of tasks, yet their mathematical analysis remains fragmented: stability and generalization are typically studied in disparate frameworks and on a case-by-case basis. Architecturally, DNNs rely on the recursive application of parametrized functions, a mechanism that can be unstable and difficult to train, making stability a primary concern. Even when training succeeds, there are few rigorous results on how well such models generalize beyond the observed data, especially in the generative setting. In this work, we leverage the theory of stochastic Iterated Function Systems (IFS) and show that two important deep architectures can be viewed as, or canonically associated with, place-dependent IFS. This connection allows us to import results from random dynamical systems to (i) establish the existence and uniqueness of invariant measures under suitable contractivity assumptions, and (ii) derive a Wasserstein generalization bound for generative modeling. The bound naturally leads to a new training objective that directly controls the collage-type approximation error between the data distribution and its image under the learned transfer operator. We illustrate the theory on a controlled 2D example and empirically evaluate the proposed objective on standard image datasets (MNIST, CelebA, CIFAR-10).

Many patterns in nature exhibit self-similarity: they can be compactly described via self-referential transformations. Said patterns commonly appear in natural and artificial objects, such as molecules, shorelines, galaxies and even images.

Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade.

There are resource-allocation problems encountered in almost every aspect of human lives: from utilities such as power systems and water systems, to transportation, and office space allocation. Many analyses of resource-allocation problems employ simplistic models of the population which ignore much of the complexity of human behaviour. Notice, for example, that the demand for a resource is often non-stationary, as exemplified by the work-day morning rush hour in transportation and the existence of predictable peaks in the demand in many other domains. Notice, further, that humans may have access to only very limited amount of information, but may still consider multiple criteria, and that their appreciation of the criteria may vary over time. As an example of a particular resource-allocation problem, we introduce a model of behaviour and the related demand process, which captures both the multi-criteria aspects of the decision making and non-stationarity of the demand process. Still, we show that the distribution of agents across resources converges in distribution, for suitable means of information provision and under certain assumptions. As our running example, we consider the problems faced by transportation authorities in charge of a road network composed of a number of road segments. For each road segment, the travel time is, in principle, a time series with a data point per a vehicle passing across the road segment.

Artificial intelligence (AI) systems often interact with multiple agents. The regulation of such AI systems often requires that {\em a priori\/} guarantees of fairness and robustness be satisfied. With stochastic models of agents' responses to the outputs of AI systems, such {\em a priori\/} guarantees require non-trivial reasoning about the corresponding stochastic systems. Here, we present an open-source PyTorch-based toolkit for the use of stochastic control techniques in modelling interconnections of AI systems and properties of their repeated uses. It models robustness and fairness desiderata in a closed-loop fashion, and provides {\em a priori\/} guarantees for these interconnections. The PyTorch-based toolkit removes much of the complexity associated with the provision of fairness guarantees for closed-loop models of multi-agent systems.

There has been much recent interest in two-sided markets and dynamics thereof. In a rather a general discrete-time feedback model, which we show conditions that assure that for each agent, there exists the limit of a long-run average allocation of a resource to the agent, which is independent of any initial conditions. We call this property the unique ergodicity. Our model encompasses two-sided markets and more complicated interconnections of workers and customers, such as in a supply chain. It allows for non-linearity of the response functions of market participants. Finally, it allows for uncertainty in the response of market participants by considering a set of the possible responses to either price or other signals and a measure to sample from these.

In power systems, one wishes to regulate the aggregate demand of an ensemble of distributed energy resources (DERs), such as controllable loads and battery energy storage systems. We suggest a notion of predictability and fairness, which suggests that the long-term averages of prices or incentives offered should be independent of the initial states of the operators of the DER, the aggregator, and the power grid. We show that this notion cannot be guaranteed with many traditional controllers used by the load aggregator, including the usual proportional-integral (PI) controller. We show that even considering the non-linearity of the alternating-current model, this notion of predictability and fairness can be guaranteed for incrementally input-to-state stable (iISS) controllers, under mild assumptions.

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network parameters $p$, a property often used to explain the advantages of DNNs over shallow NNs in approximating complicated functions. Nonetheless, a simple dimension argument shows that DNNs cannot generate all piecewise linear functions with $\ell$ linear regions as soon as $\ell > p$. It is thus natural to seek to characterize specific families of functions with $\ell$ linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) generate sequences of piecewise linear functions $F_k$ with a number of linear regions exponential in $k$. We show that, under mild assumptions, $F_k$ can be generated by a NN using only $\mathcal{O}(k)$ parameters. IFS are used extensively to generate, at low computational cost, natural-looking landscape textures in artificial images. They have also been proposed for compression of natural images, albeit with less commercial success. The surprisingly good performance of this fractal-based compression suggests that our visual system may lock in, to some extent, on self-similarities in images. The combination of this phenomenon with the capacity, demonstrated here, of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks, as well as suggest new algorithms for representing self similarities in images based on the DNN mechanism.

We consider a setting, where the output of a linear dynamical system (LDS) is, with an unknown but fixed probability, replaced by noise. There, we present a robust method for the prediction of the outputs of the LDS and identification of the samples of noise, and prove guarantees on its statistical performance. One application lies in anomaly detection: the samples of noise, unlikely to have been generated by the dynamics, can be flagged to operators of the system for further study.